An Exploratory Study of Masked Face Recognition with Machine Learning Algorithms

Automated face recognition is a widely adopted machine learning technology for contactless identification of people in various processes such as automated border control, secure login to electronic devices, community surveillance, tracking school attendance, workplace clock in and clock out. Using face masks have become crucial in our daily life with the recent world-wide COVID-19 pandemic. The use of face masks causes the performance of conventional face recognition technologies to degrade considerably. The effect of mask-wearing in face recognition is yet an understudied issue. In this paper, we address this issue by evaluating the performance of a number of face recognition models which are tested by identifying masked and unmasked face images. We use six conventional machine learning algorithms, which are SVC, KNN, LDA, DT, LR and NB, to find out the ones which perform best, besides the ones which poorly perform, in the presence of masked face images. Local Binary Pattern (LBP) is utilized as the feature extraction operator. We generated and used synthesized masked face images. We prepared unmasked, masked, and half-masked training datasets and evaluated the face recognition performance against both masked and unmasked images to present a broad view of this crucial problem. We believe that our study is unique in elaborating the mask-aware facial recognition with almost all possible scenarios including half_masked-to-masked and half_masked-to-unmasked besides evaluating a larger number of conventional machine learning algorithms compared the other studies in the literature. © 2023 IEEE.

Performance of COVID Face Mask Detection Based on SVM Algorithm Using OpenCV DNN as Preprocessing Model

As COVID-19 became a pandemic in the world, wearing a mask has become one of the best measures to prevent the spread of the epidemic, so face mask recognition in public places has become a very important part of controlling the epidemic. This paper mainly tests the performance of the OpenCV DNN preprocessing model (OpenCV DNN + SVM) based on the SVM algorithm model in the face mask recognition dataset. The dataset I use is from Kaggle called COVID Face Mask Detection Dataset. This dataset contains 503 face images with masks and 503 face images without masks. I test the performance of using OpenCV DNN + SVM and using only the SVM algorithm to evaluate this study by setting a control experimental group. In this study, it was found that using OpenCV DNN + SVM, the accuracy of ROI parameters and SVM parameters can reach 93.06% and F1score can also reach 93.06% without a lot of adjustment. The accuracy rate can only reach 68.31%, and the F1score reaches 68.31%. Findings suggest that the method using OpenCV DNN + SVM can achieve slightly better results in the COVID Face Mask Detection Dataset, and can perform better than only using the SVM algorithm. In addition, using OpenCV DNN preprocessing model based on the SVM algorithm plays an important role in feature extraction in face mask recognition. If the developer does enough parameters tuning, the accuracy will also increase. © 2022 SPIE.

Eye-based keystroke prediction for natural texts - a feasibility analysis

The use of videoconferencing is on the rise after COVID-19, being common to look at the screen and see someone typing. A side-channel attack may be launched to infer the text written from the face image. In this paper, we analyse the feasibility of such an attack, being the first proposal which work with a complete keyset (50 keys) and natural texts. We use different scenarios, lighting conditions and natural texts to increase realism. Our study involves 30 participants, who typed 49,365 keystrokes. We characterize the effect of lighting, gender, age and use of glasses. Our results show that on average 13.71% of keystrokes are revealed without error, and up to 31.8%, 52.5% and 61.2% are guessed with a maximum error of 1, 2 and 3 keys, respectively. © 2022 IEEE.

Masked Face Images Based Gender Classification using Hybrid Bat Algorithm Optimized Bagging

The face is one of the biometrics utilized to learn information from a person, such as gender. Gender classification study is expanding daily as a result of how important it is and how many other sectors, like forensics, security, business, and others, employ it. However, in order to protect themselves and stop the spread of Covid-19 during this epidemic, everyone must wear a face mask. Because many crucial facial features that help determine a person's gender are obscured by masks, using one creates an issue for the gender classification system. To obtain optimal performance outcomes, suitable hyperparameters are also required. As a result, the objective of this study is to develop a gender categorization system based on mask-covered faces utilizing a novel technique that combines several features in the Gray Level Co-occurrence Matrix (GLCM), which is then fed into the Bagging classifier.A Hybrid Bat Algorithm (HBA) is used to optimize the bagging hyperparameters. With 97% accuracy, precision, recall, and f1-score values, the suggested model is demonstrated to have greater performance than before the hyperparameters were tuned using HBA. © 2023 IEEE.

Generating Masked Facial Datasets Using Dlib-Machine Learning Library

In 2020, the COVID-19 pandemic spread globally, leading to countries imposing health restrictions on people, including wearing masks, to prevent the spread of the disease. Wearing a mask significantly decreases distinguishing ability due to its concealment of the main facial features. After the outbreak of the pandemic, the existing datasets became unsuitable because they did not contain images of people wearing masks. To address the shortage of large-scale masked faces datasets, a developed method was proposed to generate artificial masks and place them on the faces in the unmasked faces dataset to generate the masked faces dataset. Following the proposed method, masked faces are generated in two steps. First, the face is detected in the unmasked image, and then the detected face image is aligned. The second step is to overlay the mask on the cropped face images using the dlib-ml library. Depending on the proposed method, two datasets of masked faces called masked-dataset-1 and masked-dataset-2 were created. Promising results were obtained when they were evaluated using the Labeled Faces in the Wild (LFW) dataset, and two of the state-of-the-art facial recognition systems for evaluation are FaceNet and ArcFace, where the accuracy of using the two systems was 96.1 and 97, respectively with masked-dataset-1 and 87.6 and 88.9, respectively with masked-dataset-2. © 2022 IEEE.

JDSR-GAN: Constructing An Efficient Joint Learning Network for Masked Face Super-Resolution

With the growing importance of preventing the COVID-19 virus in cyber-manufacturing security, face images obtained in most video surveillance scenarios are usually low resolution together with mask occlusion. However, most of the previous face super-resolution solutions can not efficiently handle both tasks in one model. In this work, we consider both tasks simultaneously and construct an efficient joint learning network, called JDSR-GAN, for masked face super-resolution tasks. Given a low-quality face image with mask as input, the role of the generator composed of a denoising module and super-resolution module is to acquire a high-quality high-resolution face image. The discriminator utilizes some carefully designed loss functions to ensure the quality of the recovered face images. Moreover, we incorporate the identity information and attention mechanism into our network for feasible correlated feature expression and informative feature learning. By jointly performing denoising and face super-resolution, the two tasks can complement each other and attain promising performance. Extensive qualitative and quantitative results show the superiority of our proposed JDSR-GAN over some competitive methods. IEEE

Image Similarity between Masked and Unmasked Face for Consumer Electronics Applications

Face recognition has become essential as a convenient biometric-based solution for a plethora of different consumer electronics applications, including access control systems, intelligent environments, smartphone authentication systems and so on. Early in 2020, the COVID-19 pandemic caused the widespread use of face masks, which become essential for containing the outbreak. The masks cause a visible alteration in facial appearance, covering almost the 50% of the human face. In this work, an image similarity technique is applied to assess the difference between two images of the same face wearing or not wearing a face mask. Cosine Similarity measure-based Algorithm (CSA) was used to objectively infer the difficulties that modern facial recognition algorithms, based on deep learning techniques, encounter when dealing with a masked face. © 2023 IEEE.

Eye Detection from Face Images covered in Face-masks using HAAR features

The contagious illness known as COVID-19 made wearing a mask an essential part of daily life. Mask-covered faces cannot be detected by the current eye detection methods. Many biometric identification systems, like iris recognition, depend on accurate eye detection. Thus, in this study, an efficient method using machine learning for detecting eyes of people wearing mask is presented. Haar-cascade classifier is used to implement real-time eye detection from a live stream via webcam. From the live stream, frames are extracted and saved as images. Dataset was prepared by collecting face images of people wearing mask under various background. Haar-cascade classifier which was trained using 2000 positive and 4000 negative images is used to detect the position of eyes. According to the results on dataset, the system could attain an average accuracy of 96.72%. © 2022 IEEE.

Dynamic Group Difference Coding Based on Thermal Infrared Face Image for Fever Screening

Recently, noncontact temperature measurement methods based on infrared face perception have received widely attentions since fever screening plays an important role in the early prediction of respiratory infections, such as SARS, H1N1, and COVID-19. However, the performance of these methods always significantly degrades when facing the changes of environment. Thus, the majority of these methods leverage the block-body and sensors to reduce the influence of environment changes. It is a pity that the increased instrument complexity leads to higher costs and failure rate. To address the aforementioned issues, this article presents a novel fever screening method, named dynamic group difference coding (DGDC), which is based on the analysis about the influencing factors. The key idea of DGDC is to compute the temperature differences between the target person and the recently passed crowd (dynamic group). Specifically, we develop the face temperature encoder (FTE) to describe the face temperature and thus construct the difference matrix of the embedding feature between the target person and the dynamic group. Multilayer perceptions (MLP) are employed to capture the intrinsic information by characterizing the difference matrix in vertical and horizontal directions, respectively. Finally, we provide a dataset of thermal infrared face (TIF) images and conduct extensive experiments to demonstrate the advantages of the proposed method over the competing methods. © 1963-2012 IEEE.

Contrastive Representation Learning for Expression Recognition from Masked Face Images

With the worldwide spread of COVID-19, people are trying different ways to prevent the spread of the virus. One of the most useful and popular ways is wearing a face mask. Most people wear a face mask when they go out, which makes facial expression recognition become harder. Thus, how to improve the performance of the facial expression recognition model on masked faces is becoming an important issue. However, there is no public dataset that includes facial expressions with masks. Thus, we built two datasets which are a real-world masked facial expression database (VIP-DB) and a man-made masked facial expression database (M-RAF-DB). To reduce the influence of masks, we utilize contrastive representation learning and propose a two-branches network. We study the influence of contrastive learning on our two datasets. Results show that using contrastive representation learning improves the performance of expression recognition from masked face images. © 2022 ACM.

Improvement of Face Recognition Accuracy for Mask Wearers

With the spread of COVID-19, wearing masks has become mandatory in many countries around the world, and there is a need for technology to recognize facial images of people wearing masks. In this paper, we conducted experiments on learning and recognizing images of mask wearers using a method based on a pre-trained model produced by NVIDIA, and studied the recognition of face images by increasing the number of datasets using data augmentation. The experimental results show the effectiveness of the proposed method, and the proposed method is implemented on NVIDIA Jetson Xavier for real-time recognition experiments. © 2022 IEEE.

Learning 3D Face Representation with Vision Transformer for Masked Face Recognition

Masked face recognition, a non-contact biometric technology, has attracted much attention and developed rapidly during the coronavirus disease 2019 (COVID-19) outbreak. The existing work trains the masked face recognition model based on a large number of 2D masked face images. However, in practical application scenarios, it is difficult to obtain a large number of masked face images in a short period of time. Therefore, combined with 3D face recognition technology, this paper proposes a masked face recognition model trained with non-masked face images. In this paper, we locate and segment the complete face region and the face region not occluded by masks from the face point clouds. The geometric features of the 3D face surface, namely depth, azimuth, and elevation, are extracted from the above two regions to generate training data. The proposed masked face recognition model based on vision Transformer divides the complete faces and part of the faces into sequence images, and then captures the relationship between the image slices to compensate for the impact caused by the lack of face information, thereby improving the recognition performance. Comparative experiments with the state-of-the-art masked face recognition work are carried out on four databases. The experimental results show that the recognition accuracy of the proposed model is improved by 9.86% on Bosphorus database, 16.77% on CASIA-3D FaceV1 database, 2.32% on StirlingESRC database, and 34.81% on Ajmal main database, respectively, which verifies the effectiveness and stability of the proposed model. © 2022 IEEE.

Face Authentication from Masked Face Images Using Deep Learning on Periocular Biometrics

Nowadays, identity theft is an alarming issue with the growth of e-commerce and online services. Moreover, due to the Covid-19 pandemic, society has been pushed towards the usage of masks for people to safely interact with one another. It is hard to recognize a person if the face is mostly covered, even more so to artificial intelligence who have more difficulty identifying a masked individual. To further protect personal information and to develop a secure information system, more comprehensive bio-metric approaches are required. The currently used facial recognition systems are using biometrics such as periocular regions, iris, face, skin tone and racial information etc. In this paper, we apply a deep learning-based authentication approach using periocular biometric information to enhance the performance of the facial recognition system. We used the Real-World Masked Face Dataset (RMFD) and other datasets to develop our system. We implemented some experiments using CNN model on the periocular region information of the images. Hence, we developed a system that can recognize a person from only using a small region of face, which in this case is the periocular information including both eyes and eyebrows region. There is only a focus on the periocular region with our model in the view of the fact that the periocular region of the face is the main reliable source of information we can get while a person is wearing a face mask. © 2022, Springer Nature Switzerland AG.

Deep Neural Network Architecture for Face Mask Detection Against COVID-19 Pandemic Using Pre-trained Exception Network

The coronavirus disease (COVID-19) is an infectious disease caused by coronavirus. The COVID-19 virus spreads mostly through droplets of saliva or discharge from the nose when an infected person coughs or sneezes, so it is important to practice respiratory etiquette. The COVID-19 is spreading our community in a faster manner, stay safe by taking some simple precautions, such as physical distancing, wearing a mask, keeping rooms well ventilated, avoiding crowds, and cleaning hands. The appropriate use of wearing a mask is a normal part of our life. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is a novel severe acute respiratory syndrome coronavirus. Genetic variants of SARS-CoV-2 have been emerging and circulating around the world throughout the COVID-19 pandemic. To minimize the risk of transmissions, the use of face masks or coverings has been recommended in public settings. Many countries and local jurisdictions encourage or mandate the use of face masks by members of the public to limit the spread of the virus. Masks are also strongly recommended for those who may have been infected and those taking care of someone who may have the disease. In this paper, novel face mask detection on masked face data set is done by using pretrained Xception, deep learning with depth wise separable convolution. The proposed method classifies from the given face image, mask is worn or not. The proposed method is tested and validated using the face mask data set obtained from Kaggle. This data set contains about 503 face images with mask and 503 images without mask. The experimental results show that the proposed face mask detection method significantly dominates other compared pretrained models. The results of the receiver operating characteristic curve and area under curve justify the relevance of the better results in favor of the proposed method. © 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

Development of a face mask detection pipeline for mask-wearing monitoring in the era of the COVID-19 pandemic: A modular approach

During the SARS-Cov-2 pandemic, mask-wearing became an effective tool to prevent spreading and contracting the virus. The ability to monitor the mask-wearing rate in the population would be useful for determining public health strategies against the virus. In this paper, we present a two-step face mask detection approach consisting of two separate modules: 1) face detection and alignment and 2) face mask classification. This approach allows us to experiment with different combinations of face detection and face mask classification modules. More specifically, we experimented with PyramidKey and RetinaFace as face detectors while maintaining a lightweight backbone for the face mask classification module. Moreover, we also provide a relabeled annotation of the test set of the AIZOO dataset, where we rectified the incorrect labels for some face images. The evaluation results on the AIZOO and Moxa 3K datasets show that the proposed face mask detection pipeline surpassed the state-of-the-art methods. The proposed pipeline also yielded a higher mAP on the relabeled test set of the AIZOO dataset than the original test set. Since we trained the proposed model using in-the-wild face images, we can successfully deploy our model to monitor the mask-wearing rate using public CCTV images. © 2022 IEEE.

Masked Face Recognition with 3D Facial Geometric Attributes

During the coronavirus pandemic, the demand for contactless biometrics technology has promoted the development of masked face recognition. Training a masked face recognition model needs to address two crucial issues: a lack of large-scale realistic masked face datasets and the difficulty of obtaining robust face representations due to the huge difference between complete faces and masked faces. To tackle with the first issue, this paper proposes to train a 3D masked face recognition network with non-masked face images. For the second issue, this paper utilizes the geometric features of 3D face, namely depth, azimuth, and elevation, to represent the face. The inherent advantages of 3D face enhance the stability and practicability of 3D masked face recognition network. In addition, a facial geometry extractor is proposed to highlight discriminative facial geometric features so that the 3D masked face recognition network can take full advantage of the depth, azimuth and elevation information in distinguishing face identities. The experimental results on four public 3D face datasets show that the proposed 3D masked face recognition network improves the accuracy of the masked face recognition, which verifies the feasibility of training the masked face recognition model with non-masked face images. © 2022 ACM.

GAN-based Face Reconstruction for Masked-Face

Facial recognition and identification which play an important role in human-computer interaction, secure authentication and criminal face recognition, are impeded by the advent of face masks due to COVID-19 pandemic. This is a challenging problem due to the following reasons: (i) masks cover quite a large part of the face even below the chin, (ii) it is not possible to collect and prepare a real paired-face images with and without mask object, (iii) face alterations and the presence of different masks is even more challenging. In this work, we propose a general framework that can be used to reconstruct the hidden part of face concealed by mask. We have employed GAN-based unpaired domain translation technique to translate masked face images from the source to the unmasked images in the destination domain. To this end, we also create a paired datasets of real face images and synthesized correspondence's with face-masks and use it towards training of our proposed GAN-based facial reconstruction system which can be used for facial identification and secure authentication in human-computer interaction. The obtained results demonstrate that our model outperforms other representative state-of-the-art face completion approaches both qualitatively and quantitatively. © 2022 Owner/Author.

Deepfake Detection for Fake Images with Facemasks

Hyper-realistic face image generation and manipulation have given rise to numerous unethical social issues, e.g., invasion of privacy, threat of security, and malicious political maneuvering, which resulted in the development of recent deepfake detection methods with the rising demands of deepfake forensics. Proposed deepfake detection methods to date have shown remarkable detection performance and robustness. However, none of the suggested deepfake detection methods assessed the performance of deepfakes with the facemask during the pandemic crisis after the outbreak of the COVID-19. In this paper, we thoroughly evaluate the performance of state-of-The-Art deepfake detection models on the deepfakes with the facemask. Our result shows that fake facial images with facemask can deceive well-known deepfake detection models, thereby evading the real-world security systems. © 2022 ACM.

Detection of Face Mask in Thermal Images Using Deep CNN

Coronavirus (COVID-19) is a major health crisis across the globe, and one of the effective methods against the transmission of virus is wearing a face mask. It has become extremely important to monitor if people are wearing face mask when moving out. The aim of project is to automate the detection of face mask using images captured from a thermal camera. The problem is posed as a binary classification problem, and the input face image is classified as with mask or without mask. Transfer learning is used for classification, wherein deep CNN model, MobileNetV2, is used as a base model for feature extraction. A dataset of face images with and without mask is prepared using lepton FLIR camera interfaced to a Raspberry Pi board. The built model is able to detect people who are wearing a face mask and not wearing with an accuracy of 98%. © 2022, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

Information Technology for Person Identification by Occluded Face Image

This paper describes the research of information technology for person identification by occluded face image. As far as the spread of coronavirus disease (COVID-19) raised the problem of identification by facial image with masks covering the face as a prevention measure, the research of face recognition and identification technologies has become crucial for all of the cybersecurity areas based on the identity verification by digital technologies. The proposed algorithm, that is the cornerstone of the information technology, is based on the methods of anisotropic diffusion for image preprocessing, Gabor wavelet transform, histogram of oriented gradients (HOG) and local binary patterns in 1-dimensional space (1DLBP) for image feature vector extraction, and square Euclidean distance metric for vector classification. Experiments on the proposed technology after applying it on the occluded images from the SCface database provided the result of 85%, increased on 2.5% after image format and resolution conversion. © 2022 IEEE.